Iterative data analysis enabled through query result abstraction

ABSTRACT

The present invention is generally directed to a system, method and article of manufacture for accessing data represented abstractly through an abstraction model. In one embodiment, a data repository abstraction layer provides a logical view of an underlying data repository that is independent of the particular manner of data representation. For each successive query, the data repository abstraction layer is replaced or redefined to provide a restricted logical view of the underlying data repository.

CROSS-RELATED APPLICATIONS

This application contains similar subject matter as that described inU.S. patent application Ser. No. 10/131,984, entitled “REMOTE DATAACCESS AND INTEGRATION OF DISTRIBUTED DATA SOURCES THROUGH DATA SCHEMAAND QUERY ABSTRACTION” and assigned to International Business Machines,Inc.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention generally relates to data processing, and moreparticularly, to the accessing data through a logical framework.

2. Description of the Related Art

Databases are computerized information storage and retrieval systems.The most prevalent type of database is the relational database, atabular database in which data is defined so that it can be reorganizedand accessed in a number of different ways. A relational databasemanagement system (DBMS) is a database management system that usesrelational techniques for storing and retrieving data.

Regardless of the particular architecture, in a DBMS, a requestingentity (e.g., an application, the operating system or a user) demandsaccess to a specified database by issuing a database access request.Such requests may include, for instance, simple catalog lookup requestsor transactions and combinations of transactions that operate to read,change and add specified records in the database. These requests aremade using high-level query languages such as the Structured QueryLanguage (SQL). Illustratively, SQL is used to make interactive queriesfor getting information from and updating a database such asInternational Business Machines' (IBM) DB2, Microsoft's SQL Server, anddatabase products from Oracle, Sybase, and Computer Associates. The term“query” denominates a set of commands for retrieving data from a storeddatabase. Queries take the form of a command language that letsprogrammers and programs select, insert, update, find out the locationof data, and so forth.

In many cases, particularly in research-oriented environments, the taskof analyzing information is typically a multi-step process involvinggeneration of an initial set of query results that is further reducedthrough subsequent queries. For example, in a medical researchenvironment, an initial query could be posed to find a set of researchcandidates meeting a particular diagnosis or test result profile. In alarge data warehouse environment, it may take a lot of time andresources to process a query of this nature, in particular if thecriteria specified is complex. Once the query results are returned, itmay be desirable to apply other criteria to further reduce or subset theresults returned. While this could be accomplished by extending theoriginal query with additional logic, it would be more efficient toperform the subsequent query against the results of the original query.Unfortunately, current methods to address this scenario are typicallyrather manual and error prone. Specifically, the user is required to gothrough several steps to save the results from the original query in aform that would allow for subsequent queries to be performed against it.In a relational environment, this would require a table to be created tohold the results of the original query. Similarly, in an XML-basedrepository, query result data would need to be stored in the repositoryin order to be re-queried. In either case, the user would then need tounderstand the physical schema used to represent the initial queryresults and expresses subsequent query taking this schema into account.

Another problem encountered in the iterative query scenario is that asubsequent query may not return a desired set of results. The user maythen wish to go back to a prior point in the progression of intermediatequery results. Using conventional techniques the user must consciouslysave the results from each query in order to re-establish a prior queryiteration.

Therefore, what is needed is a query framework providing flexibility indata analysis.

SUMMARY OF THE INVENTION

The present invention provides a method, system and article ofmanufacture for accessing physical data through an abstraction model.The abstraction model includes metadata describing and defining aplurality of logical fields.

In one aspect, a method of using a logical framework to query data isprovided. The method includes at least defining a logical view of thedata; and iteratively restricting the logical view of the data for eachsuccessive abstract query, based on at least one result criterion of animmediately previous abstract query.

In another aspect, a method for constructing abstract queries defined bylogical fields is provided. The method includes at least providing aninitial abstract data model defining a plurality of logical fieldsmapped to physical data having a particular schema; receiving a firstabstract query comprising at least one condition and a result criterioncomprising at least one of the plurality of logical fields; transformingthe first abstract query into an executable query with reference to theinitial abstract data model; executing the executable query; returningresults produced by execution of the executable query; and generatinganother abstract data model comprising only the at least one of theplurality of logical fields of the result criterion, wherein the atleast one of the plurality of logical fields is mapped to the results.

In another aspect, a method for constructing abstract queries defined bylogical fields is provided. The method includes at least providing aninitial abstract data model defining a plurality of logical fields andmapping each logical field to physical data; receiving a first abstractquery comprising at least two logical fields defined by the initialabstract data model; transforming the first abstract query into anexecutable query with reference to the initial abstract data model;executing the executable query; returning results produced by executionof the executable query; and generating another abstract data model.Generating another abstract data model includes, for each of the atleast two logical fields of the first abstract query, (i) retrieving,from the initial abstract data model, a logical field definition for thelogical field; (ii) updating the logical field definition to refer to aportion of the results; and (iii) adding the updated logical fielddefinition to the another abstract data model.

Yet another aspect provides a computer-readable medium containing aprogram which, when executed by a processor, performs an operation foreach abstract query in a succession of abstract queries. The operationincludes at least initializing an abstract data model; adding to theabstract data model only those one or more logical fields defined asresult fields in the abstract query; mapping each logical field of theabstract data model to a different portion of results returned for theabstract query as a result of being executed; making the abstract datamodel available for construction of a next abstract query in thesuccession of abstract queries, whereby logical fields defined by agiven abstract data model are limited to those defined as result fieldsin a last-executed abstract query of the succession of abstract queries.

Still another aspect provides a computer comprising a memory and atleast one processor, and further comprising a framework for defining andprocessing abstract queries for accessing physical data. The frameworkincludes at least an abstract data model generator configured togenerate abstract data models based on (i) a logical result field of alast-executed abstract query in a series of abstract queries; and (ii)results returned for the last-executed abstract query. Each abstractdata model defines a logical view of the data and includes at least (a)a logical field definition only for each logical result field of thelast-executed abstract query; and (b) mapping information for each ofthe one or more logical field definitions, which maps each of thelogical field definitions to a separate portion of the results returnedfor the last-executed abstract query.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the manner in which the above recited features, advantages andobjects of the present invention are attained and can be understood indetail, a more particular description of the invention, brieflysummarized above, may be had by reference to the embodiments thereofwhich are illustrated in the appended drawings.

It is to be noted, however, that the appended drawings illustrate onlytypical embodiments of this invention and are therefore not to beconsidered limiting of its scope, for the invention may admit to otherequally effective embodiments.

FIG. 1 is a block diagram of an illustrative computer architecture.

FIGS. 2A and 2B are relational views of software components of oneembodiment of the invention.

FIG. 3 is a flow chart illustrating the operation of a runtimecomponent.

FIG. 4 is a flow chart illustrating the operation of a runtimecomponent.

FIG. 5 is a block diagram showing a data repository abstractiongenerator configured to generate abstract data models mapping logicalfields to results of an immediately previous query.

FIG. 6 is a block diagram illustrating the generation of an abstractdata model based on results of an abstract query.

FIG. 7 is a block diagram illustrating the generation of anotherabstract data model based on results of an abstract query.

FIG. 8 it is a flow chart illustrating an algorithm to create anabstract data model.

FIGS. 9-11 are screens of a user interface configured for constructingabstract queries based on displayed logical fields, each of which isdefined by an abstract data model.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Introduction

One embodiment of the invention is implemented as a program product foruse with a computer system and described below. The program(s) of theprogram product defines functions of the embodiments (including themethods described herein) and can be contained on a variety ofsignal-bearing media. Illustrative signal-bearing media include, but arenot limited to: (i) information permanently stored on non-writablestorage media (e.g., read-only memory devices within a computer such asCD-ROM disks readable by a CD-ROM drive); (ii) alterable informationstored on writable storage media (e.g., floppy disks within a diskettedrive or hard-disk drive); or (iii) information conveyed to a computerby a communications medium, such as through a computer or telephonenetwork, including wireless communications. The latter embodimentspecifically includes information downloaded from the Internet and othernetworks. Such signal-bearing media, when carrying computer-readableinstructions that direct the functions of the present invention,represent embodiments of the present invention.

In general, the routines executed to implement the embodiments of theinvention, may be part of an operating system or a specific application,component, program, module, object, or sequence of instructions. Thesoftware of the present invention typically is comprised of a multitudeof instructions that will be translated by the native computer into amachine-readable format and hence executable instructions. Also,programs are comprised of variables and data structures that eitherreside locally to the program or are found in memory or on storagedevices. In addition, various programs described hereinafter may beidentified based upon the application for which they are implemented ina specific embodiment of the invention. However, it should beappreciated that any particular nomenclature that follows is used merelyfor convenience, and thus the invention should not be limited to usesolely in any specific application identified and/or implied by suchnomenclature.

In one embodiment, a particular data definition framework (also referredto herein as a data repository abstraction (DRA) layer/component/modelor abstract data model) is provided for accessing (e.g., querying andmodifying) data independent of the particular manner in which the datais physically represented. The DRA includes metadata describing anddefining a plurality of logical fields which map to physical data. Foreach iterative query, a DRA is derived based on the results of thequery. The subsequent query is then executed based on the most currentderived DRA.

Physical View of Environment

FIG. 1 depicts a block diagram of a networked system 100 in whichembodiments of the present invention may be implemented. In general, thenetworked system 100 includes a client (i.e., generally any requestingentity such as a user or application) computer 102 (three such clientcomputers 102 are shown) and at least one server computer 104 (one suchserver computer 104 is shown). The client computer 102 and the servercomputer 104 are connected via a network 126. In general, the network126 may be a local area network (LAN) and/or a wide area network (WAN).In a particular embodiment, the network 126 is the Internet. However, itis noted that aspects of the invention need not be implemented in adistributed environment. As such, the client computers 102 and theserver computer 104 are more generally representative of any requestingentity (such as a user or application) issuing queries and a receivingentity configured to handle the queries, respectively.

The client computer 102 includes a Central Processing Unit (CPU) 110connected via a bus 130 to a memory 112, storage 114, an input device116, an output device 119, and a network interface device 118. The inputdevice 116 can be any device to give input to the client computer 102.For example, a keyboard, keypad, light-pen, touch-screen, track-ball, orspeech recognition unit, audio/video player, and the like could be used.The output device 119 can be any device to give output to the user,e.g., any conventional display screen. Although shown separately fromthe input device 116, the output device 119 and input device 116 couldbe combined. For example, a display screen with an integratedtouch-screen, a display with an integrated keyboard, or a speechrecognition unit combined with a text speech converter could be used.

The network interface device 118 may be any entry/exit device configuredto allow network communications between the client computer 102 and theserver computer 104 via the network 126. For example, the networkinterface device 118 may be a network adapter or other network interfacecard (NIC).

Storage 114 is preferably a Direct Access Storage Device (DASD).Although it is shown as a single unit, it could be a combination offixed and/or removable storage devices, such as fixed disc drives,floppy disc drives, tape drives, removable memory cards, or opticalstorage. The memory 112 and storage 114 could be part of one virtualaddress space spanning multiple primary and secondary storage devices.

The memory 112 is preferably a random access memory sufficiently largeto hold the necessary programming and data structures of the invention.While the memory 112 is shown as a single entity, it should beunderstood that the memory 112 may in fact comprise a plurality ofmodules, and that the memory 112 may exist at multiple levels, from highspeed registers and caches to lower speed but larger DRAM chips.

Illustratively, the memory 112 contains an operating system 124.Illustrative operating systems, which may be used to advantage, includeLinux and Microsoft's Windows®. More generally, any operating systemsupporting the functions disclosed herein may be used.

The memory 112 is also shown containing a browser program 122 that, whenexecuted on CPU 110, provides support for navigating between the variousservers 104 and locating network addresses at one or more of the servers104. In one embodiment, the browser program 122 includes a web-basedGraphical User Interface (GUI), which allows the user to display HyperText Markup Language (HTML) information. More generally, however, thebrowser program 122 may be any GUI-based program capable of renderingthe information transmitted from the server computer 104.

The server computer 104 may be physically arranged in a manner similarto the client computer 102. Accordingly, the server computer 104 isshown generally comprising a CPU 130, a memory 132, and a storage device134, coupled to one another by a bus 136. Memory 132 may be a randomaccess memory sufficiently large to hold the necessary programming anddata structures that are located on the server computer 104.

The server computer 104 is generally under the control of an operatingsystem 138 shown residing in memory 132. Examples of the operatingsystem 138 include IBM OS/400®, UNIX, Microsoft Windows®, and the like.More generally, any operating system capable of supporting the functionsdescribed herein may be used.

The memory 132 further includes one or more applications 140 and anabstract query interface 146. The applications 140 and the abstractquery interface 146 are software products comprising a plurality ofinstructions that are resident at various times in various memory andstorage devices in the computer system 100. When read and executed byone or more processors 130 in the server 104, the applications 140 andthe abstract query interface 146 cause the computer system 100 toperform the steps necessary to execute steps or elements embodying thevarious aspects of the invention. The applications 140 (and moregenerally, any requesting entity, including the operating system 138and, at the highest level, users) issue queries against a database.Illustrative sources against which queries may be issued include localdatabases 156 ₁ . . . 156 _(N), and remote databases 157 ₁ . . . 157_(N), collectively referred to as database(s) 156-157). Illustratively,the databases 156 are shown as part of a database management system(DBMS) 154 in storage 134. More generally, as used herein, the term“databases” refers to any collection of data regardless of theparticular physical representation. By way of illustration, thedatabases 156-157 may be organized according to a relational schema(accessible by SQL queries) or according to an XML schema (accessible byXML queries). However, the invention is not limited to a particularschema and contemplates extension to schemas presently unknown. As usedherein, the term “schema” generically refers to a particular arrangementof data which is described by a data repository abstraction 148.

In one embodiment, the queries issued by the applications 140 aredefined according to an application query specification 142 includedwith each application 140. The queries issued by the applications 140may be predefined (i.e., hard coded as part of the applications 140) ormay be generated in response to input (e.g., user input). In eithercase, the queries (referred to herein as “abstract queries”) arecomposed using logical fields defined by the abstract query interface146. In particular, the logical fields used in the abstract queries aredefined by the data repository abstraction (DRA) component 148 of theabstract query interface 146. The abstract queries are executed by aruntime component 150 which transforms the abstract queries into a form(referred to herein as a concrete query) consistent with the physicalrepresentation of the data contained in one or more of the databases156-157. The queries may be configured to access the data and returnresults, or to modify (i.e., insert, delete or update) the data. Theapplication query specification 142, the abstract query interface 146and the data repository abstraction component 148 are further describedwith reference to FIGS. 2A-B.

In one embodiment, elements of a query are specified by a user through agraphical user interface (GUI). The content of the GUIs is generated bythe application(s) 140. In a particular embodiment, the GUI content ishypertext markup language (HTML) content which may be rendered on theclient computer systems 102 with the browser program 122. Accordingly,the memory 132 includes a Hypertext Transfer Protocol (http) serverprocess 138 (e.g., a web server) adapted to service requests from theclient computer 102. For example, the process 138 may respond torequests to access a database(s) 156, which illustratively resides onthe server 104. Incoming client requests for data from a database156-157 invoke an application 140. When executed by the processor 130,the application 140 causes the server computer 104 to perform the stepsor elements embodying the various aspects of the invention, includingaccessing the database(s) 156-157. In one embodiment, the application140 comprises a plurality of servlets configured to build GUI elements,which are then rendered by the browser program 122. Where the remotedatabases 157 are accessed via the application 140, the data repositoryabstraction component 148 is configured with a location specificationidentifying the database containing the data to be retrieved. Thislatter embodiment will be described in more detail below.

In one embodiment, the server computer 104 is further configured with adata repository abstraction generator 164 (DRA generator). The DRAgenerator 164 is invoked to generate data repository abstractioncomponents or to modify the existing data repository abstractioncomponent. For example, DRA generator may generate modified versions ofthe data repository abstraction component 148, either by replacing thedata repository abstraction component or changing its attributes. Suchmodified versions are also referred to herein as “derived DRAs” becausethe modified versions are derived from a preexisting DRA. In oneembodiment, a derived DRA persists in memory only for a user session.For example, a derived DRA may be discarded once a user logs off of thesystem. Alternatively, a derived DRA may be stored and retrieved forlater use.

FIG. 1 is merely one hardware/software configuration for the networkedclient computer 102 and server computer 104. Embodiments of the presentinvention can apply to any comparable hardware configuration, regardlessof whether the computer systems are complicated, multi-user computingapparatus, single-user workstations, or network appliances that do nothave non-volatile storage of their own. Further, it is understood thatwhile reference is made to particular markup languages, including HTML,the invention is not limited to a particular language, standard orversion. Accordingly, persons skilled in the art will recognize that theinvention is adaptable to other markup languages as well as non-markuplanguages and that the invention is also adaptable future changes in aparticular markup language as well as to other languages presentlyunknown. Likewise, the http server process 138 shown in FIG. 1 is merelyillustrative and other embodiments adapted to support any known andunknown protocols are contemplated.

Logical/Runtime View of Environment

FIGS. 2A-B show a plurality of interrelated components of the invention.The requesting entity (e.g., one of the applications 140) issues a query202 as defined by the respective application query specification 142 ofthe requesting entity. The resulting query 202 is generally referred toherein as an “abstract query” because the query is composed according toabstract (i.e., logical) fields rather than by direct reference to theunderlying physical data entities in the databases 156-157. As a result,abstract queries may be defined that are independent of the particularunderlying data representation used. In one embodiment, the applicationquery specification 142 may include both criteria used for dataselection (selection criteria 204) and an explicit specification of thefields to be returned (return data specification 206) based on theselection criteria 204.

The logical fields specified by the application query specification 142and used to compose the abstract query 202 are defined by the datarepository abstraction component 148. In general, the data repositoryabstraction component 148 exposes information as a set of logical fieldsthat may be used within a query (e.g., the abstract query 202) issued bythe application 140 to specify criteria for data selection and specifythe form of result data returned from a query operation. The logicalfields are defined independently of the underlying data representationbeing used in the databases 156-157, thereby allowing queries to beformed that are loosely coupled to the underlying data representation.The data to which logical fields of the DRA 148 are mapped may belocated in a single repository (i.e., source) of data or a plurality ofdifferent data repositories. Thus, the DRA 148 may provide a logicalview of one or more underlying data repositories. By using an abstractrepresentation of a data repository, the underlying physicalrepresentation can be more easily changed or replaced without affectingthe application making the changes. Instead, the abstract representationis changed with no changes required by the application. In addition,multiple abstract data representations can be defined to supportdifferent applications against the same underlying database schema thatmay have different default values or required fields.

In general, the data repository abstraction component 148 comprises aplurality of field specifications 208 ₁, 208 ₂, 208 ₃, 208 ₄ and 208 ₅(five shown by way of example), collectively referred to as the fieldspecifications 208. Specifically, a field specification is provided foreach logical field available for composition of an abstract query. Eachfield specification comprises a logical field name 210 ₁, 210 ₂, 210 ₃,210 ₄, 210 ₅ (collectively, field name 210) and an associated accessmethod 212 ₁, 212 ₂, 212 ₃, 212 ₄, 212 ₅ (collectively, access method212). The access methods associate (i.e., map) the logical field namesto a particular physical data representation 214 ₁, 214 ₂ . . . 214 _(N)in a database (e.g., one of the databases 156) according to parametersreferred to herein as physical location parameters. By way ofillustration, two data representations are shown, an XML datarepresentation 214 ₁ and a relational data representation 214 ₂.However, the physical data representation 214 _(N) indicates that anyother data representation, known or unknown, is contemplated.

Any number of access methods are contemplated depending upon the numberof different types of logical fields to be supported. In one embodiment,access methods for simple fields, filtered fields and composed fieldsare provided. The field specifications 208 ₁, 208 ₂ and 208 ₅ exemplifysimple field access methods 212 ₁, 212 ₂, and 212 ₅, respectively.Simple fields are mapped directly to a particular entity in theunderlying physical data representation (e.g., a field mapped to a givendatabase table and column). By way of illustration, the simple fieldaccess method 212 ₁ shown in FIG. 2B maps the logical field name 210 ₁(“FirstName”) to a column named “f_name” in a table named “contact”,where the table name and the column name are the physical locationparameters of the access method 212 ₁. The field specification 208 ₃exemplifies a filtered field access method 212 ₃. Filtered fieldsidentify an associated physical entity and provide rules used to definea particular subset of items within the physical data representation. Anexample is provided in FIG. 2B in which the filtered field access method212 ₃ maps the logical field name 210 ₃ (“AnytownLastName”) to aphysical entity in a column named “I_name” in a table named “contact”and defines a filter for individuals in the city of Anytown. Anotherexample of a filtered field is a New York ZIP code field that maps tothe physical representation of ZIP codes and restricts the data only tothose ZIP codes defined for the state of New York. The fieldspecification 208 ₄ exemplifies a composed field access method 212 ₄.Composed access methods compute a logical field from one or morephysical fields using an expression supplied as part of the accessmethod definition. In this way, information which does not exist in theunderlying data representation may computed. In the example illustratedin FIG. 2B the composed field access method 212 ₃ maps the logical fieldname 210 ₃ “AgelnDecades” to “AgeInYears/10”. Another example is a salestax field that is composed by multiplying a sales price field by a salestax rate.

It is noted that the data repository abstraction component 148 shown inFIG. 2B is merely illustrative of selected logical field specificationsand is not intended to be comprehensive. As such, the abstract query 202shown in FIG. 2B includes some logical fields for which specificationsare not shown in the data repository abstraction component 148, such as“State” and “Street”.

It is contemplated that the formats for any given data type (e.g.,dates, decimal numbers, etc.) of the underlying data may vary.Accordingly, in one embodiment, the field specifications 208 include atype attribute which reflects the format of the underlying data.However, in another embodiment, the data format of the fieldspecifications 208 is different from the associated underlying physicaldata, in which case an access method is responsible for returning datain the proper format assumed by the requesting entity. Thus, the accessmethod must know what format of data is assumed (i.e., according to thelogical field) as well as the actual format of the underlying physicaldata. The access method can then convert the underlying physical datainto the format of the logical field.

By way of example, the field specifications 208 of the data repositoryabstraction component 148 shown in FIG. 2A are representative of logicalfields mapped to data represented in the relational data representation214 ₂. However, other instances of the data repository abstractioncomponent 148 map logical fields to other physical data representations,such as XML. Further, in one embodiment, a data repository abstractioncomponent 148 is configured with access methods for procedural datarepresentations. One embodiment of such a data repository abstractioncomponent 148 is described below with respect to FIG. 8.

An illustrative abstract query corresponding to the abstract query 202shown in FIG. 2 is shown in Table I below. By way of illustration, thedata repository abstraction 148 is defined using XML. However, any otherlanguage may be used to advantage.

TABLE I QUERY EXAMPLE 001 <?xml version=“1.0”?> 002 <!--Query stringrepresentation: (FirstName = “Mary“ AND LastName = 003 “McGoon”) ORState = “NC”--> 004 <QueryAbstraction> 005  <Selection> 006 <ConditioninternalID=“4”> 007  <Condition field=“FirstName”  operator=“EQ”value=“Mary” 008 internalID=“1”/> 009  <Condition field=“LastName” operator=“EQ” value=“McGoon” 010 internalID=“3”relOperator=“AND”></Condition> 011 </Condition> 012 <Conditionfield=“State” operator=“EQ” value=“NC” internalID=“2” 013relOperator=“OR“></Condition> 014  </Selection> 015  <Results> 016 <Field name=“FirstName”/> 017  <Field name=“LastName”/> 018  <Fieldname=“State”/> 019  </Results> 020 </QueryAbstraction>Illustratively, the abstract query shown in Table I includes a selectionspecification (lines 005-014) containing selection criteria and aresults specification (lines 015-019). In one embodiment, a selectioncriterion consists of a field name (for a logical field), a comparisonoperator (=, >, <, etc) and a value expression (what is the field beingcompared to). In one embodiment, result specification is a list ofabstract fields that are to be returned as a result of query execution.A result specification in the abstract query may consist of a field nameand sort criteria.

An illustrative instance of a data repository abstraction component 148corresponding to the abstract query in Table I is shown in Table IIbelow. By way of illustration, the data repository abstraction component148 is defined using XML. However, any other language may be used toadvantage.

TABLE II DATA REPOSITORY ABSTRACTION EXAMPLE 001 <?xml version=“1.0”?>002 <DataRepository> 003  <Category name=“Demographic”> 004 <Fieldqueryable=“Yes” name=“FirstName” displayable=“Yes”> 005 <AccessMethod>006 <Simple columnName=“f_name” tableName=“contact”></Simple> 007</AccessMethod> 008 <Type baseType=“char”></Type> 009 </Field> 010<Field queryable=“Yes” name=“LastName” displayable=“Yes”> 011<AccessMethod> 012 <Simple columnName=“l_name”tableName=“contact”></Simple> 013 </AccessMethod> 014 <TypebaseType=“char”></Type> 015 </Field> 016 <Field queryable=“Yes”name=“State” displayable=“Yes”> 017 <AccessMethod> 018 <SimplecolumnName=“state” tableName=“contact”></Simple> 019 </AccessMethod> 020<Type baseType=“char”></Type> 021 </Field> 022  </Category> 023</DataRepository>

Note that lines 004-009 correspond to the first field specification 208₁ of the DRA 148 shown in FIG. 2B and lines 010-015 correspond to thesecond field specification 208 ₂. For brevity, the other fieldspecifications defined in Table I have not been shown in FIG. 2B. Notealso that Table I illustrates a category, in this case “Demographic”. Acategory is a grouping of one or more logical fields. In the presentexample, “First Name”, “Last Name” and “State” are logical fieldsbelonging to the common category, “Demographic”.

In any case, a data repository abstraction component 148 contains (orrefers to) at least one access method which maps a logical field tophysical data. However, the foregoing embodiments are merelyillustrative and the logical field specifications may include a varietyof other metadata. In one embodiment, the access methods are furtherconfigured with a location specification defining a location of the dataassociated with the logical field. In this way, the data repositoryabstraction component 148 is extended to include description of amultiplicity of data sources that can be local and/or distributed acrossa network environment. The data sources can be using a multitude ofdifferent data representations and data access techniques. In thismanner, an infrastructure is provided which is capable of capitalizingon the distributed environments prevalent today. One approach foraccessing a multiplicity of data sources is described in more detail inU.S. patent application Ser. No. 10/131,984, entitled “REMOTE DATAACCESS AND INTEGRATION OF DISTRIBUTED DATA SOURCES THROUGH DATA SCHEMAAND QUERY ABSTRACTION” and assigned to International Business Machines,Inc.

FIG. 3 shows an illustrative runtime method 300 exemplifying oneembodiment of the operation of the runtime component 150. The method 300is entered at step 302 when the runtime component 150 receives as inputan instance of an abstract query (such as the abstract query 202 shownin FIG. 2). At step 304, the runtime component 150 reads and parses theinstance of the abstract query and locates individual selection criteriaand desired result fields. At step 306, the runtime component 150 entersa loop (comprising steps 306, 308, 310 and 312) for processing eachquery selection criteria statement present in the abstract query,thereby building a data selection portion of a Concrete Query. In oneembodiment, a selection criterion consists of a field name (for alogical field), a comparison operator (=, >, <, etc) and a valueexpression (what is the field being compared to). At step 308, theruntime component 150 uses the field name from a selection criterion ofthe abstract query to look up the definition of the field in the datarepository abstraction 148. As noted above, the field definitionincludes a definition of the access method used to access the physicaldata associated with the field. The runtime component 150 then builds(step 310) a Concrete Query Contribution for the logical field beingprocessed. As defined herein, a Concrete Query Contribution is a portionof a concrete query that is used to perform data selection based on thecurrent logical field. A concrete query is a query represented inlanguages like SQL and XML Query and is consistent with the data of agiven physical data repository (e.g., a relational database or XMLrepository). Accordingly, the concrete query is used to locate andretrieve data from a physical data repository, represented by thedatabases 156-157 shown in FIG. 1. The Concrete Query Contributiongenerated for the current field is then added to a Concrete QueryStatement. The method 300 then returns to step 306 to begin processingfor the next field of the abstract query. Accordingly, the processentered at step 306 is iterated for each data selection field in theabstract query, thereby contributing additional content to the eventualquery to be performed.

After building the data selection portion of the concrete query, theruntime component 150 identifies the information to be returned as aresult of query execution. As described above, in one embodiment, theabstract query defines a list of abstract fields that are to be returnedas a result of query execution, referred to herein as a resultspecification. A result specification in the abstract query may consistof a field name and sort criteria. Accordingly, the method 300 enters aloop at step 314 (defined by steps 314, 316, 318 and 320) to add resultfield definitions to the concrete query being generated. At step 316,the runtime component 150 looks up a result field name (from the resultspecification of the abstract query) in the data repository abstraction148 and then retrieves a Result Field Definition from the datarepository abstraction 148 to identify the physical location of data tobe returned for the current logical result field. The runtime component150 then builds (as step 318) a Concrete Query Contribution (of theconcrete query that identifies physical location of data to be returned)for the logical result field. At step 320, Concrete Query Contributionis then added to the Concrete Query Statement. Once each of the resultspecifications in the abstract query has been processed, the query isexecuted at step 322.

One embodiment of a method 400 for building a Concrete QueryContribution for a logical field according to steps 310 and 318 isdescribed with reference to FIG. 4. At step 402, the method 400 querieswhether the access method associated with the current logical field is asimple access method. If so, the Concrete Query Contribution is built(step 404) based on physical data location information and processingthen continues according to method 300 described above. Otherwise,processing continues to step 406 to query whether the access methodassociated with the current logical field is a filtered access method.If so, the Concrete Query Contribution is built (step 408) based onphysical data location information for some physical data entity. Atstep 410, the Concrete Query Contribution is extended with additionallogic (filter selection) used to subset data associated with thephysical data entity. Processing then continues according to method 300described above.

If the access method is not a filtered access method, processingproceeds from step 406 to step 412 where the method 400 queries whetherthe access method is a composed access method. If the access method is acomposed access method, the physical data location for each sub-fieldreference in the composed field expression is located and retrieved atstep 414. At step 416, the physical field location information of thecomposed field expression is substituted for the logical fieldreferences of the composed field expression, whereby the Concrete QueryContribution is generated. Processing then continues according to method300 described above.

If the access method is not a composed access method, processingproceeds from step 412 to step 418. Step 418 is representative of anyother access methods types contemplated as embodiments of the presentinvention. However, it should be understood that embodiments arecontemplated in which less then all the available access methods areimplemented. For example, in a particular embodiment only simple accessmethods are used. In another embodiment, only simple access methods andfiltered access methods are used.

As described above, it may be necessary to perform a data conversion ifa logical field specifies a data format different from the underlyingphysical data. In one embodiment, an initial conversion is performed foreach respective access method when building a Concrete QueryContribution for a logical field according to the method 400. Forexample, the conversion may be performed as part of, or immediatelyfollowing, the steps 404, 408 and 416. A subsequent conversion from theformat of the physical data to the format of the logical field isperformed after the query is executed at step 322. Of course, if theformat of the logical field definition is the same as the underlyingphysical data, no conversion is necessary.

Derived Data Repository Abstraction Components

In one embodiment, a data repository abstraction component is derivedfor each iterative query based on the results of the executed query.That is, the derived DRA includes a logical field for each result fieldof the query, wherein each logical field is mapped to the appropriateunderlying physical data. The subsequent query is then executed based onthe most current derived DRA.

Referring now to FIG. 5, a first DRA 504A is shown. The DRA 504Aincludes a plurality of logical fields (not shown) which map to data ina database 506. An abstract query 502A issued against the DRA 504A isconfigured with conditions 501A and return criteria 503A. In an SQLcounterpart statement (after the abstract query 502A is transformed intoa concrete query), the return criteria 503 are the SELECT clause and theconditions are the WHERE clause of the statement. When executed, thequery 502A returns a first result set 508A (which may be formatted bythe runtime component 150, shown in FIG. 2A). Using the first DRA 504Aand the first result set 508A as input, the DRA generator 162 generatesa second DRA 504B configured with a plurality of logical fields (notshown) mapped to the first result set 508A. A second abstract query 502Bis then executed with respect to the second DRA 504B which produces asecond result set 508B. The DRA generator 162 may then generate anotherDRA which maps to the second result set 508B. This process may beperformed for each successive abstract query as represented by the Nthabstract query 502N having conditions 501N and return criteria 503N. TheNth query 502N is issued with respect to the Nth DRA 504N which maps tothe result set 508N -1 of the previous query.

As noted previously, it may be desirable for a user to return to theprevious derived DRA. As such, one embodiment provides for preservingthe DRAs using a stack 510. A stack is a well-known data storage area orbuffer, which may be implemented as a push-down list following a LIFO(last-in first-out) scheme. That is, the last item pushed onto (i.e.,placed on) the stack is the first item popped (i.e., removed) from thestack. For each derivation of a DRA, the previous DRA is pushed onto thestack 510. A user desiring to return to a previous derivation of the DRAsuccessively pops each DRA from the stack 510 until arriving at thedesired DRA. In addition, prior results may be saved, e.g., in a seriesof temporary tables with each table containing the results of a givenprior query. Each derived DRA then references the temporary tablecontaining results of the query used to generate the derived DRA. Eachtemporary table may be maintained for as long a period of time as thecorresponding derived DRA is maintained (e.g. if derived DRAs persistfor the duration of the user's session, so would the correspondingtemporary tables).

Referring now to FIG. 6, additional details for the generation of DRAsare described. Illustratively, FIG. 6 shows an initial DRA 608. The DRA608 includes a plurality of field specifications 610 each of which mapto data in a data repository 618. In the present example, the datarepository 618 is a relational database including a plurality of tables620A-D (four shown). Accordingly, the field specifications 610 aredefined by a logical field name 612, a table name 614 and a column name616. As described above with respect to FIG. 2B, the values of the tablename 614 and column name 616 are the parameters passed to the accessmethod of the respective logical field in order to retrieve data fromthe repository 618. Illustratively, the logical fields also includecategory metadata, whereby the logical fields 1-6 are part of Category 1and logical fields 7-10 are part of Category 2.

A first abstract query 602 is shown configured with a condition 604 andreturn criteria 606. The condition 604 and return criteria 606 of theabstract query 602 are each configured with one or more logical fields,each of which are represented in the DRA 608. By way of example, thecondition 604 specifies a value of “5” for Field 4 and the returncriteria 606 specifies that data is to be returned for Fields 2-4 andField 7 where the condition 604 is satisfied.

Execution of the abstract query 602 returns query results 622. The queryresults are arranged as a data structure consistent with the particularschema of the data repository 618. Accordingly, where the datarepository 618 is a relational database, the query results 622 areformatted as a table, T1. Alternatively, since the user is shielded fromthe particulars of the underlying data, the query results 622 may evenbe formatted in a schema different from that of the data repository 618.For example, if the data repository 618 is a relational database, thequery results 622 may be formatted as an XML document. It iscontemplated that the latter approach would require either a databaseengine that supported both relational and XML queries or more than onedatabase engine, with one engine handling relational data storage andqueries and the other handling XML data storage and queries.

In any case, the DRA generator 162 generates a derived DRA 624 based onthe results 622. Specifically, the derived DRA includes a plurality oflogical fields 626 each of which maps to the query results 622. Notethat the plurality of logical fields 626 is a subset of the originalplurality of logical fields 608 and corresponds to the logical fieldsspecified in the return criteria 606 of the abstract query 602. However,the physical location parameters (e.g., table name 626 and column name628 in the case of a relational schema) of the logical fields 626 havebeen updated to point to the query results 622. Specifically, eachlogical field 626 points to a different column of the table T1.

In one aspect, the derived DRA 624 preserves attributes of the originalDRA 608. For example, the data type, the logical field name and thecategorization scheme represented in the original DRA 608 are preservedin the derived DRA 624. Attributes of the underlying data repository 618may also be preserved.

Referring to the FIG. 7 a second query iteration based on the derivedDRA 624 of FIG. 6 is illustrated. In this case, a second query 702includes a condition 704 specifying the logical Field 2, and resultcriteria 706 specifying the logical Fields 2-3. Consequently, the queryresults 708 include a column corresponding to each logical fieldspecified in the result criteria. In the present example, the logicalfields specified in the result criteria 706 map to column A1 of table T1and column CK′ of table T1, respectively. Accordingly, the results 708populate a table T2 having a column A1 and a column CK′. The DRAgenerator 162 then generates a derived DRA 710 which includes thelogical fields corresponding to column A1 of table T1 and column CK′ oftable T1, except that the logical fields are updated to map to each ofthe two columns, A1 and CK′, in the results 708.

Referring now to FIG. 8, a flowchart illustrates one embodiment of amethod 800 implemented by the DRA generator 162 for creating a derivedDRA. The method 800 may be entered after execution of the abstract query802 based on an initial DRA 810 and for which results 822 were returned.The DRA generator 162 then initializes a derived DRA 806 to empty (step804). A loop is then entered (and step 808) for each result field in theresult criteria of the abstract query 802. If the metadata of theinitial DRA 810 specifies a category (or categories), the category(s)for the current result field is retrieved from the initial DRA 810 (step812) and added to the derived DRA 806 (step 814). Next, the logicalfield definition for the current result field is retrieved from theinitial DRA 810 (step 816). The DRA generator 162 then updates theaccess method for the logical field definition to refer to the results822 (step 818) and as the logical field definition to the derived DRA806 (step 820).

In the embodiment described with respect to FIG. 8, a derived DRA is aseparate entity, i.e., separate from the initial DRA which may persistin some data storage area (e.g., the stack 510 described with respectFIG. 5). Alternatively, the derived DRA may be a modified version of theinitial DRA by removing the appropriate logical fields from the initialDRA. That is, those fields which are not present as result fields in theimmediately preceding query are removed.

In one aspect, an advantage is provided to users constructing queriesagainst data repository abstraction models. Since the view of theunderlying data is dictated by the DRA, the user is given anincreasingly restricted view of the data with each successive query.This aspect of the invention may be illustrated with reference to FIGS.9-11.

Referring first to FIG. 9, a screen 900 is shown which may be viewedthrough the browser program 122 (FIG. 1). The screen 900 includes aSelection panel 902, a Query Conditions panel 904 in a Query ResultsFields panel 906. The Selection panel 902 displays each of the logicalfields defined by a DRA. If defined, categories are also displayed.Using the displayed logical fields, the user may construct a query byspecifying conditions in the Query Conditions panel 904 and specifyingresult fields in the Query Results Fields panel 906, as illustrated bythe screen 900 FIG. 10. In this particular illustration, the QueryResults Fields panel 906 includes six logical fields (result fields):Gender, State, Glucose Test, Test Date, First Name and Last Name. Thequery may then be executed by clicking the Submit the 908. Afterexecution of the query, the results of the query may be displayed to theuser in another screen (not shown).

For the next query, the Selection panel 902 shows the logical fields ofthe derived DRA generated by the DRA generator 162 based on the queryresults fields of the previous query. For the illustrative query of FIG.10, the logical fields of the derived DRA are shown in the Selectionpanel 902 of FIG. 11. Accordingly, the displayed logical fields includeFirst Name, Last Name, Gender, State, Test Date and Glucose Test. Basedon the displayed logical fields, the user may construct another query.This process may be performed iteratively whereby the number ofdisplayed/available logical fields is successively reduced.Additionally, at any point during the iterative query constructionprocess the user may decide to return to a previous instance of the DRAby clicking a Back button 1102. As noted above, navigation in thismanner may be facilitated by the provision of a stack, although anyother technique may be used to this end.

In various embodiments, numerous advantages over the prior art areprovided. In one aspect, advantages are achieved by defining a loosecoupling between the application query specification and the underlyingdata representation. Rather than encoding an application with specifictable, column and relationship information, as is the case where SQL isused, the application defines data query requirements in a more abstractfashion that are then bound to a particular physical data representationat runtime. The loose query-data coupling of the present inventionenables requesting entities (e.g., applications) to function even if theunderlying data representation is modified or if the requesting entityis to be used with a completely new physical data representation thanthat used when the requesting entity was developed. In the case with agiven physical data representation is modified or restructured, thecorresponding data repository abstraction is updated to reflect changesmade to the underlying physical data model. The same set of logicalfields are available for use by queries, and have merely been bound todifferent entities or locations in physical data model. As a result,requesting entities written to the abstract query interface continue tofunction unchanged, even though the corresponding physical data modelhas undergone significant change. In the event a requesting entity is tobe used with a completely new physical data representation differentthan that used when the requesting entity was developed, the newphysical data model may be implemented using the same technology (e.g.,relational database) but following a different strategy for naming andorganizing information (e.g., a different schema). The new schema willcontain information that may be mapped to the set of logical fieldsrequired by the application using simple, filtered and composed fieldaccess method techniques. Alternatively, the new physical representationmay use an alternate technology for representing similar information(e.g., use of an XML based data repository versus a relational databasesystem). In either case, existing requesting entities written to use theabstract query interface can easily migrate to use the new physical datarepresentation with the provision of an alternate data repositoryabstraction which maps fields referenced in the query with the locationand physical representation in the new physical data model.

In another aspect, the ease-of-use for the application builder and theend-user is facilitated. Use of an abstraction layer to representlogical fields in an underlying data repository enables an applicationdeveloper to focus on key application data requirements without concernfor the details of the underlying data representation. As a result,higher productivity and reduced error rates are achieved duringapplication development. With regard to the end user, the datarepository abstraction provides a data filtering mechanism, exposingpertinent data and hiding nonessential content that is not needed by aparticular class end-user developing the given query.

Solutions implementing the present model use the provided abstract queryspecification to describe its information requirements, without regardfor the location or representation of the data involved. Queries aresubmitted to the runtime component which uses the data repositoryabstraction component to determine the location and method used toaccess each logical piece of information represented in the query. Inone embodiment, the runtime component also includes the aforementioneddata caching function to access the data cache.

In one aspect, this model allows solutions to be developed independentof the physical location or representation of the data used by thesolution, making it possible to easily deploy the solution to a numberof different data topologies and allowing the solution to function incases where data is relocated or reorganized over time. In anotheraspect, this approach also simplifies the task of extending a solutionto take advantage of additional information. Extensions are made at theabstract query level and do not require addition of software that isunique for the location or representation of the new data beingaccessed. This method provides a common data access method for softwareapplications that is independent of the particular method used to accessdata and of the location of each item of data that is referenced. Thephysical data accessed via an abstract query may be representedrelationally (in an existing relational database system), hierarchically(as XML) or in some other physical data representation model. Amultitude of data access methods are also supported, including thosebased on existing data query methods such as SQL and XQuery and methodsinvolving programmatic access to information such as retrieval of datathrough a Web Service invocation (e.g., using SOAP) or HTTP request.

While the foregoing is directed to embodiments of the present invention,other and further embodiments of the invention may be devised withoutdeparting from the basic scope thereof, and the scope thereof isdetermined by the claims that follow.

1. A computer-implemented method of using a logical framework to querydata, comprising: displaying a logical view of data in a user interface,wherein the logical view includes a plurality of logical fields definedby a first data abstraction model, comprising: a plurality of logicalfield definitions for each of the plurality of logical fields, whereinat least some of the plurality of logical field definitions include alogical field name and a reference to an access method for retrievingdata from at least one physical field of the data, whereby the retrieveddata is exposed as a value of the respective logical field name; anditeratively restricting the logical view of the data for each successiveabstract query, based on at least one result criterion of an immediatelyprevious abstract query; wherein the at least one result criterionspecifies one or more fields for which a query result was returned forthe immediately previous abstract query, and wherein the immediatelyprevious abstract query further includes selection criteria specifyingone or more conditions to be satisfied by the query result: whereiniteratively restricting the logical view of the data comprises making aportion of the data exposed by the plurality of logical fieldsunavailable through the logical view by performing at least one of:removing all values of the data corresponding to the result criterion;and preserving only those values of the data corresponding to the resultcriterion, and removing all other values of the data.
 2. The method ofclaim 1, wherein iteratively restricting the logical view comprisessubsetting the plurality of logical fields relative to the immediatelyprevious abstract query.
 3. The method of claim 1, wherein the datacomprises a plurality of tables in a database.
 4. The method of claim 1,wherein the result criterion is a logical field.
 5. The method of claim1, wherein iteratively restricting the logical view comprises generatinga second data abstraction model without elements of the first dataabstraction model.
 6. The method of claim 5, wherein generating thesecond data abstraction model comprises mapping logical fields definedby the second data abstraction model to the query result of theimmediately previous abstract query.
 7. The method of claim 1, whereiniteratively restricting the logical view comprises generating a seconddata abstraction model with only a portion of the plurality of logicalfield definitions contained in the first data abstraction model.
 8. Themethod of claim 7, wherein generating the second data abstraction modelcomprises mapping the portion of the plurality of logical fields to thequery result of the immediately previous abstract query.
 9. Acomputer-implemented method of using a logical framework to query data,comprising: defining a logical view of the data, wherein the logicalview includes a plurality of logical fields defined by a dataabstraction model, comprising: a plurality of logical field definitionsfor each of the plurality of logical fields, wherein at least some ofthe plurality of logical field definitions include a logical field nameand a reference to an access method for retrieving data from at leastone physical field of data, whereby the retrieved data is exposed as avalue of the respective logical field name; and iteratively restrictingthe logical view of the data for each successive abstract query, basedon at least one result criterion of an immediately previous abstractquery; wherein the at least one result criterion specifies one or morefields for which a query result was returned for the immediatelyprevious abstract query, and wherein the immediately previous abstractquery further includes selection criteria specifying one or moreconditions to be satisfied by the query result: wherein iterativelyrestricting the logical view comprises removing a portion of the dataabstraction model and mapping a remaining portion of the dataabstraction model to the query result of the immediately previousabstract query.
 10. A computer-implemented method of using a logicalframework to query data, comprising: displaying a logical view of datain a user interface, wherein the logical view includes a plurality oflogical fields defined by a data abstraction model, comprising: aplurality of logical field definitions for each of the plurality oflogical fields, wherein at least some of the plurality of logical fielddefinitions include a logical field name and a reference to an accessmethod for retrieving data from the at least one physical field of thedata, whereby the retrieved data is exposed as a value of the respectivelogical field name; and iteratively restricting the logical view of thedata for each successive abstract query, based on at least one resultcriterion of an immediately previous abstract query; wherein the atleast one result criterion specifies one or more fields for which aquery result was returned for the immediately previous abstract query,and wherein the immediately previous abstract query further includesselection criteria specifying one or more conditions to be satisfied bythe query result; wherein iteratively restricting the logical viewcomprises removing elements of the data abstraction model and preservingonly those elements of the data abstraction model corresponding to theat least one result criterion of the immediately previous abstractquery.